80 research outputs found
Alpha Entanglement Codes: Practical Erasure Codes to Archive Data in Unreliable Environments
Data centres that use consumer-grade disks drives and distributed
peer-to-peer systems are unreliable environments to archive data without enough
redundancy. Most redundancy schemes are not completely effective for providing
high availability, durability and integrity in the long-term. We propose alpha
entanglement codes, a mechanism that creates a virtual layer of highly
interconnected storage devices to propagate redundant information across a
large scale storage system. Our motivation is to design flexible and practical
erasure codes with high fault-tolerance to improve data durability and
availability even in catastrophic scenarios. By flexible and practical, we mean
code settings that can be adapted to future requirements and practical
implementations with reasonable trade-offs between security, resource usage and
performance. The codes have three parameters. Alpha increases storage overhead
linearly but increases the possible paths to recover data exponentially. Two
other parameters increase fault-tolerance even further without the need of
additional storage. As a result, an entangled storage system can provide high
availability, durability and offer additional integrity: it is more difficult
to modify data undetectably. We evaluate how several redundancy schemes perform
in unreliable environments and show that alpha entanglement codes are flexible
and practical codes. Remarkably, they excel at code locality, hence, they
reduce repair costs and become less dependent on storage locations with poor
availability. Our solution outperforms Reed-Solomon codes in many disaster
recovery scenarios.Comment: The publication has 12 pages and 13 figures. This work was partially
supported by Swiss National Science Foundation SNSF Doc.Mobility 162014, 2018
48th Annual IEEE/IFIP International Conference on Dependable Systems and
Networks (DSN
Self-Repairing Disk Arrays
As the prices of magnetic storage continue to decrease, the cost of replacing
failed disks becomes increasingly dominated by the cost of the service call
itself. We propose to eliminate these calls by building disk arrays that
contain enough spare disks to operate without any human intervention during
their whole lifetime. To evaluate the feasibility of this approach, we have
simulated the behavior of two-dimensional disk arrays with n parity disks and
n(n-1)/2 data disks under realistic failure and repair assumptions. Our
conclusion is that having n(n+1)/2 spare disks is more than enough to achieve a
99.999 percent probability of not losing data over four years. We observe that
the same objectives cannot be reached with RAID level 6 organizations and would
require RAID stripes that could tolerate triple disk failures.Comment: Part of ADAPT Workshop proceedings, 2015 (arXiv:1412.2347
Reducing the Energy Footprint of a Distributed Consensus Algorithm
Abstract—The Raft consensus algorithm is a new distrib-uted consensus algorithm that is both easier to understand and more straightforward to implement than the older Paxos algo-rithm. Its major limitation is its high energy footprint. As it relies on majority consensus voting for deciding when to commit an update, Raft requires five participants to protect against two simultaneous failures. We propose two methods for reducing this huge energy footprint. Our first proposal consists of adjusting Raft quorums in a way that would allow updates to proceed with as few as two servers while requiring a larger quorum for electing a new leader. Our second proposal consists of replacing one or two of the five Raft servers with witnesses, that is, lightweight servers that maintain the same metadata as other servers but hold no data and can therefore run on very low-power hosts. We show that these substitutions have little impact on the cluster availability but very different impacts on the risks of incurring a data loss. Keywords-Distributed computing; Fault-tolerant computing
INVOLVING CLIENTS IN THE DISTRIBUTION OF VIDEOS ON DEMAND
We present a stream tapping protocol that involves clients in the video distribution process. As in conventional stream tapping, our protocol lets new clients tap the most recent broadcast of the video they are watching. While conventional stream tapping required the server to send to these clients the part of the video they missed, our protocol delegates this task to the clients that are already watching the video, thus greatly reducing the workload of the server. Unlike previous solutions involving clients in the video distribution process, our protocol works with clients that can only upload video data at a fraction of the video consumption rate and includes a mechanism to control its network bandwidth consumption
The Case for Aggressive Partial Preloading in Broadcasting Protocols for Video-on-Demand
Broadcasting protocols for video-on-demand usually consume over fifty percent of their bandwidth to distribute the first ten to fifteen minutes of the videos they distribute. Since all these protocols require the user set-top box to include a disk drive, we propose to use this drive to store the first five to twenty minutes of the ten to twenty most popular videos. This will provide low-cost instant access to these videos. 1
LIMITING THE CLIENT BANDWIDTH OF BROADCASTING PROTOCOLS FOR VIDEOS ON DEMAND
pagoda broadcasting. Broadcasting protocols can lower the cost of video-ondemand services by more efficiently distributing all videos that are simultaneously watched by many viewers. The most efficient broadcasting protocols require a customer settop box capable of capturing data from five to seven video channels at the same time. We show how to modify existing broadcasting protocols so that their client bandwidth would never exceed three to four channels and apply our method to the fast broadcasting and the new pagoda broadcasting protocols. Our data show that this modification has only a moderate effect on the overall performance of the two protocols because their server bandwidth never increases by more than 15 percent. 1
- …